Proposal to

	    THE NATIONAL SCIENCE FOUNDATION

		         for



	     ADVANCED AUTOMATION DEVELOPMENT

		         by

   THE STANFORD ARTIFICIAL INTELLIGENCE LABORATORY


  Jerome Feldman, Associate Professor of Computer Science
  Associate Director, Artificial Intelligence Laboratory
	      Principal Investigator

  Thomas Binford, Research Associate in Computer Science
	      Associate Investigator

  Richard Paul, Research Associate in Computer Science
	      Associate Investigator




		       February 1973
	







	      Computer Science Department

	     School of Humanities and Sciences


	       STANFORD UNIVERSITY

	      Stanford, California

		        ABSTRACT


The Stanford Artificial Intelligence Laboratory proposes to undertake
a  program in advanced automation over a period of one year at a cost
of $143,391.  This  proposal  is  to  continue  and  extend  programs
sponsored under ARPA Contract SD-183.

This  proposal  is  for  initial  study and preliminary research on a
three  year  "proof  of  concept"  project  in  computer   controlled
automatic  assembly.  The  outline of the project has been worked out
with industrial and governmental advice, but more analysis as well as
more research is needed.

The  project includes a pilot assembly station carrying out practical
assembly tasks, a hardware-software system ready for production,  and
fundamental  work  on the underlying problems of vision, manipulation
and problem-solving. We believe the laboratory is uniquely  qualified
to  carry out this program which will be an important contribution to
increasing industrial productivity.

Our basic approach involves  a  general  purpose  system  integrating
sensors  and  manipulators  controlled by sophisticated programs in a
digital computer. We have worked with such  systems  since  1965  and
have  developed  many  of the basic ideas required in general purpose
assembly devices. The main thrust of this proposal  is  to  establish
the  practicality  of  our  methods  and carry their development to a
point from which industry can take over.

The research program  is  divided  into  four  main  sections:  Pilot
assembly  station,  vision development, manipulation development, and
systems support. Each section contains a  brief  description  of  the
laboratory's past work in the area, how it is to be applied, and what
extensions are to be undertaken. Finally, we  consider  a  number  of
important  issues  concerning  our  relations  with industry and with
other reseach and development efforts in advanced automation.


       TABLE OF CONTENTS

Part								Page

I	The Need for Advanced Automation ...................... 
II	Research Plan ......................................... 
	A.	The Pilot Assembly Station .................... 
	B.	Research in Vision ............................ 
	C.	Manipulator Development ....................... 
	D.	Systems Requirements for Advanced Automation .. 
III	Dissemination of Results and Relation with Industry ... 
IV	Staff ................................................. 
V	Computing Facilities .................................. 
VI	Budget ................................................ 
VII	References ............................................ 
VIII	Relevant Laboratory Publications ...................... 
IX	Films ................................................. 
X	Cognizant Personnel ................................... 

I. THE NEED FOR ADVANCED AUTOMATION


The  contributions  that  advanced  automation  can  make  to enhance
productivity in manufacturing are  now  widely  appreciated.  Current
interest  in  advanced  computer-oriented  automation  stems from two
sources.  The first is the decreasing cost  of  computation  combined
with an improved understanding of how to apply computation to various
aspects of manufacturing. The second cause is  the  realization  that
many   current   manufacturing  operations,  primarily  assembly  and
inspection, will require complex automation  methods  which  are  now
research  techniques. The same considerations apply to the automation
of tasks such as laboratory and medical procedures, but the  emphasis
has  been (as ours will be) on manufacturing. A recent ARPA-sponsored
study [Anderson 1972] in which we participated suggests the following
major benefits from advanced automation:
	improved national productivity
	reduced prices
	increased product quality and uniformity
	customization of products
	increased job satisfaction. 
We would like to add just one comment. As we  have  seen  with  other
technological  developments such as numerical-controlled tools, there
will be things made which were previously impractical to make.  There
is  widespread  agreement in the technical, industrial and government
communities on the potential benefits of advanced  automation.  There
is  much  less  agreement  on  the  level of sophistication needed in
automation hardware and software. While there are  large  application
areas  in  which  specially  constructed  machines  or simple general
purpose  devices  will  prove  optimal,  we  believe  that  the  most
practical  solution for intermediate quantities (500 to 100,000) will
be  to  develop  general  purpose  assembly  and  inspection  devices
controlled by sophisticated programs in digital computers.

Several  studies  have  indicated some priorities among directions of
automation. The ISI study [Anderson] indicates that less than 20%  of
discrete  manufacturing  is amenable to special purpose automation. A
Chrysler Corporation study indicates  that  car  assembly  labor  has
first  priority  in automation; assembly plant direct labor is 40% of
direct corporate labor. It was felt that automation was essential  in
the small parts division. The ISI study quotes a figure of 75% as the
fraction of batch produced discrete manufactures. One  conclusion  is
that  "the emphasis in development should be on simple vision devices
and on low-bandwidth tactile and force sensing feedback". That  study
indicates that the assembly process typically accounts for 50 percent
of production cost. Although such  estimates  are  crude,  the  study
indicates  a  potential  savings of $27 billion on manufactures which
total $492 billion.

We assume that present manipulators have a small but  stable  market,
and  that  most  will  have minicomputers soon (Sundstrand does now).
Will the incremental cost of  touch,  force  and  visual  sensors  be
offset  by  the  opening  of  new  capabilities, and will the cost be
offset by reduced cost of system  design  for  machines  which  allow
loose  alignment tolerances? We estimate that touch and force sensing
will add about 5% to the price of current manipulators of the Unimate
class.   We  estimate  that  by  1977, the incremental cost of visual
sensors, a laser ranging device such as  we  use  now,and  additional
memory  requirements,  will  add about 15%-20% to current manipulator
prices. If labor costs continue to rise, part of the increase will be
offset,  but  cost  comparisons will not change appreciably over that
time scale unless significant new mechanical  designs  are  made.   A
conclusion is that sensor-equipped robots will be available at slight
additional cost. We explore the benefits of incremental  improvements
to manipulators.

We  do not expect mechanical production costs to drop as dramatically
as electronics costs fall with integration. The most  promising  cost
reduction  development would be high-production standard modules such
as robots.  As an example, consider the  Stanford  arm;  produced  in
unit   quantity,   parts,   manufacturing   and  assembly  costs  are
approximately $10K. The arm is much simpler than an automobile; there
is  no  reason why it should cost more. A smaller, more recent design
sells for $3500 in quantities of two.

A robot with two arms, minicomputer control, force and touch  sensors
could  be  constructed  for  under $10K in quantities of 1000 now. To
summarize: two arms (new small Scheinman design) $2000 with hand; arm
drivers  and  arm  interface electronics $1000; force and touch $800;
small computer 8k  memory  $4500;  total  $8300.  Assuming  that  the
computer  price falls to $1200 by 1978, the cost is $5000. Of course,
these are small arms, suited for parts assembly, not automotive  body
assembly.   The  larger  arms,  assuming  slight  redesign  for  cost
consciousness, would cost about $2500 in units of 1000, for  a  total
system  cost (not price) of $12300, falling to $9000 by 1978. This is
the range of price of present transfer machines  which  have  limited
market  because  they  have  limited motion, no control, and are thus
locked out of the economy of scale. There is a dynamic which leads to
economy  of  scale;  the  greater  the  versatility, the wider is the
market for applications. This will  be  true  particularly  in  small
production  and  job  shop  operations  which  have been difficult to
automate.  The larger the production, the more  versatility  for  the
same  initial  cost  as  special  devices.   Also,  with high volume,
standardization of maintenance, and availability of parts and service
are improved. The quality of the device is also improved.

The  economies of scale in production of assembly modules are matched
by economies of intellectual  scale  in  development  of  design  for
automation.    The   design   process   is   modularized  and  easily
communicable; there is  a  common  discipline  which  can  be  widely
absorbed; developments of many sources can be integrated. Programming
and training done once for one machine  can  be  transferred  to  all
machines  in  a  plant,  and  even transferred nationwide. Functional
modularity has been emphasized in our studies. Modularity is  clearly
possible  and  there  is  a  powerful  economic incentive to assemble
widely varying systems from one, two, or three arm modules, force and
touch   modules,  vision  and  ranging  modules,  with  planning  and
monitoring modules. This would also entail software  modularity  with
computer-aided  mechanical design and job specification for robots on
a local or network time-sharing service.

The alternative is special design for each operation. We expect  that
at  some  point,  design  will be cheaper with standard robot modules
than with special purpose designs, just as it is now more  economical
to  use  mini-computer  controllers than special purpose controllers.
Modularity has the  potential  to  reduce  design  time  and  improve
predictability  of  design time and costs, permitting better planning
and  response  to  competition.  To  some  extent,  the   dehumanized
industrial  worker  now fills the role of standard module, so that we
do not expect discontinuous change. The continuity  with  decades  of
experience  with  design  for  human  versatility  in assembly can be
relied upon;  robots  with  some  versatility,  sensing,  contingency
actions,  and  dexterity  will  provide a transition to newer schemes
which will evolve more slowly. Another advantage of  general  purpose
techniques is the ease with which the size of the robot can be scaled
up or down. There appears to be no difficulty in making very large or
very small assembly stations which use essentially the same programs.

Versatile  assembly  devices  have  an efficiency advantage in medium
scale and job shop production; their utilization  efficiency  can  be
higher  because  versatility  allows  a  variety  of operations to be
scheduled  at  one  station.   The  availability  of  computers   for
production  scheduling  will  allow  the  flexibility  to  be used to
advantage. Robots would also have longer life over which to  amortize
costs,  since  they  could  function  with greatly changed production
operations, and evolve with new hardware and software  modules.  They
would  have  higher  reliability,  since  they would have monitoring,
contingency action, and error recovery. They would be  self-aligning,
self-calibrating, and self-testing.

With  simple  devices,  inspection  and  assembly  operations must be
broken down into small operations with transfer  operations  between.
This introduces extra transfer devices with orientation-preserving or
re-orienting steps, and overlapping operations which can be  combined
in  an  advanced  device. The extra cost of jigs and fixtures, and of
stopping  assembly  lines  adds  to  the  cost  of  special   purpose
automation.  Robots  can  cut  down  on  real  estate,  since several
operations can be done at one station.

Because of the adaptibility of work stations, a quicker  response  to
market  and  competition is possible. Production runs can be smaller,
with less capital tied up in inventory and  unused  machines.   There
can be better response to the needs of high technology industry which
often requires  small  quantities.   There  has  been  difficulty  in
meeting   the   tight   tolerances   necessary   for   operation   of
non-intelligent devices.  It is hard to estimate the value of systems
which work within current industrial tolerance, rather than requiring
radical improvements in engineering  practice,  but  that  continuity
does seem important.

Serious issues arise in the development of robots with simple sensory
systems. Some evaluation issues should be settled  by  demonstration,
since  much  sharper  analyses  can  be made this way. We propose to:
demonstrate practical speed and dexterity  with  two  arms  and  tool
using.  Demonstrate  the  generality  of arm control and planning for
various arms  and  devices.  Demonstrate  designs  which  standardize
interface  with  devices and which lower interface costs. Demonstrate
programming of assembly devices in  an  interactive  APT-like  system
(develop  the  Stanford Assembly Robot Management System, ARMS); show
the commonality of operations aong similar assembly tasks.

There is already a significant degree of  industrial  involvement  in
computer-aided  assembly.  GM  Research  Lab has demonstrated a wheel
mounting operation using visual alignment.  IBM  has  a  small  robot
project.  Hitachi  has  demonstrated  a robot with touch and a system
with visual identification of objects on a conveyor belt, with visual
control  of  manipulation.  Can  industry  be  expected to adequately
support this development?  Development  of  primitive  robots  is  an
evolutionary  process,  not  a single product development. Thus it is
not possible to keep secret and  maintain  an  enormous  productivity
advantage;  the  techniques  are  usable by everyone. The area is too
big; the payoffs are huge and the investment level  is  large.  Also,
traditionally, industry has been dependent on suppliers. But with the
exception of IBM, the suppliers are smaller and less able to risk the
development.  Management  techniques  require  inordinate  return  on
capital. An alternative support could come again from  the  military.
The  ARPA study and the history of numerical controlled machine tools
are  natural  consequences  of  the  vast  military  procurement   in
technologically advanced products. In the case of N/C, funds could be
diverted from an outmoded technology to development of devices  which
were  essential  to  production  plans  for  the  future.  It is high
technology products which are most competitive internationally in the
civilian  economy.  The  record of government investment in numerical
control, solid  state  electronics,  computers,  and  electro-optical
devices has been very productive. However, in large scale development
of civilian products and international markets,  foreign  governments
have  taken  the  lead.  The  development of general purpose assembly
devices will benefit manufacturers in a broad spectrum  of  industry,
particularly  medium  scale  manufacturing. It is an appropriate area
for governmental action to help meet a national need.

Given sensor-equipped robots at little extra  cost,  they  will  have
visual  ability  to  sense  color,  strong  contrasts  in brightness,
recognize known objects from two dimensional outlines  (top  view  of
flat  object) or three dimensionally using depth, to a certain extent
they will be  able  to  recognize  three  dimensional  objects  using
monocular  recognition.  They  will  be able to visually align mating
parts  in  assembly  operations,  increase   reliability   by   self-
calibration  and  flaw  and  disaster  detection,  and  permit  large
tolerances in parts transfer, even permitting parts to be taken  from
a  bin.   Several  of  these  abilities  are not yet available in the
laboratory,  but  with  known  conditions  and  objects,   and   some
engineering  (e.g.  color coding) are achievable within the time span
of the proposed project.

The major determinant of the economic impact of assembly robots  will
be  the  extent  of their capabilities. Assessing and extending these
capabilities is the principle goal of the proposed project.


II. RESEARCH PLAN

	A. THE PILOT ASSEMBLY STATION

The basic goal of the proposed project  is  to  implement  a  general
purpose  assembly  station  and  to  prove  its applicability to real
problems. The project will be considered a success  if  one  or  more
enterprises  chooses  to  manufacture  and market the general purpose
station. There are a number of hardware and software problems  to  be
solved in meeting our goal.

In one configuration, the assembly station is assumed to be connected
to a large time-shared computer facility. Not only does  this  appear
to  be a good design for our problem, but it is also necessary for an
integrated  programmed  automation  facility.  The  only   emphasized
statement  in  the ARPA study report [Anderson 1972] is: "The maximum
impact  of  computers  in  manufacturing  will  come  from  complete,
real-time  computer  cognizance  and  control  of  all  processes and
resources, allowing the precise scheduling and  allocation  of  these
resources."

Each  assembly  station will consist of a small digital computer, one
or more manipulators and cameras. Manipulators and  cameras  will  be
combined  as  modules  in  systems  to suit a given task, and will be
digitally controlled by the station computer.  The  station  computer
may be connected to a central computer or may stand alone. The latter
configuration is less efficient in  the  long  run,  but  allows  for
incremental automation.

The program in the station computer will be flexible as to the number
and position of manipulators and cameras. It will control input  from
the  cameras,  provide for some pre-processing of the picture and the
application of simple tests to the  visual  data.   In  the  case  of
manipulators  it will provide for simultaneous servo control. It will
also process  other  inputs,  such  as  from  touch  sensors  on  the
manipulators,  from photocell interrupt devices, limit switches, etc.
Both analog and binary outputs will be available for the  control  of
other devices.

Each  station  computer will have access to a task program specifying
the actions and tests it should perform to carry out its  task.  This
task  program  will  be compiled by the central computer. Compilation
will consist of the analysis of predicted scenes in order to  specify
simple tests that the station computer can carry out, the calculation
of collision free trajectories for the manipulators, the sequence  of
operations  and  simple  contingency  plans. etc. We are planning for
tasks  whose  normal  execution  does  not  require  detailed   scene
analysis.  If  scene analysis becomes necessary (e.g. a dropped part)
we envision the station sending an image to the central computer  for
processing.

The  central  computer will perform overall control of the individual
assembly stations and the relative rates of  production.  If  at  any
station  a  planned test fails, for which a contingency plan does not
exist, or any unexpected sensor signals  are  received,  the  central
computer  will  also be interrupted and appropriate actions taken. If
the assembly station is not connected to a  central  computer,  these
operations will require human intervention.


Most of the hardware required  for  the  assembly  station  has  been
developed  for  the  laboratory.  Manipulators  and  cameras  will be
discussed in detail below. The major additional problem is to  refine
the   existing   designs   to  meet  the  demands  of  an  industrial
environment.

We are now acquiring small computer  such  as  the  PDP-11  and  will
interface  the existing arms to it together with one or more cameras.
A software package for the small computer will then be  developed  in
order to control the manipulators, read the TV cameras and to perform
low level visual analysis. Programs  to  plan  tasks,  calibrate  the
system  and  to  perform higher level visual analysis, error recovery
and strategy planning will be written in a high  level  language  for
the time-sharing processor.

The most difficult part of the assembly station, and the heart of the
project is a demonstration of the feasibility of the station. We plan
to do this by developing programs which actually assemble and inspect
a variety of fairly complex manufactured items.

A typical example is our current effort, assembly  of  an  automobile
water  pump.  The  major  parts  are  to  be  located, the screws are
supplied by a feeder, as is standard. Locating the body and cover  of
the  pump  require  a gross localization, then a fine localization of
screw  holes.  Touch  provides  a  final  accuracy  and   independent
coordination  of the hand. The placing of alignment pins within screw
holes could be monitored visually; although in this case,the previous
actions   provide  adequate  accuracy.  Placing  of  the  cover  over
alignment pins can be carried  out  without  vision.  Improved  force
sensing now under development will be useful for that operation.

As  we move to more difficult tasks, the sophistication required goes
up quite rapidly.  The  current  manipulation  routines  seem  to  be
expandable  to cover anticipated problems, especially since we intend
to employ special purpose manipulators (tools) for  some  tasks.  The
main part of our effort will be vision for assembly; an assembly task
will be performed using vision, touch, and  two  hands.   The  vision
effort  will  be  integrated with other abilities, and while no great
generality will be stressed, new modules will  be  incorporated  into
the  system,  and  a set of techniques will evolve. These will depend
heavily on techniques similar to  those  of  our  recent  studies  in
visual  feedback  and hand/eye coordination [Gill 1972]. The internal
model of the world will be  used  to  predict  visual  appearance  of
selected  areas  of  the  scene, and a set of verification procedures
will confirm or contradict the predictions. Many  of  the  operations
will  be  connected  with  supplying  feedback  to  manipulation, for
alignment and insertion. These visual procedures will have models for
the parts and approximate locations.

This and other visual tasks connected with assembly will initially be
performed as special  case  problems.  In  addition,  to  make  these
techniques   available,   we  will  pursue  development  of  strategy
programs. This step is essential if intelligent assembly devices  are
to  be conveniently programmed for complex tasks. Within the division
of labor between large and small machines, this facility must  reside
in  the  central  time-sharing  facility which has large memory, file
storage, and problem-solving languages. The system must have a  model
of  the  parts  for  assembly,and  a  sequence  of  instructions  for
assembly. It will be necessary to have a representation of shape  and
models for usual assembly operations such as insertion and alignment,
and the ability to solve simple problems  in  connection  with  these
operations,  all  this  to  enable  convenient communication with the
user. When these facilities exist, the addition of new  features  and
operations  can be incorporated smoothly, so that users can call on a
growing library of planning modules. The result of a planning  effort
is  a  cycle  of  machine  planning and man-machine interaction which
results in a plan  network  for  the  mini  computer  which  controls
individual machines.


	B.  RESEARCH  IN  VISION  

Research in computer vision has progressed so that the larger part of
our  effort is directed toward vision in real scenes. We outline past
research, including work on visual feedback,  development  of  useful
modules,  analysis  of  some outdoor scenes, interpretation of scenes
with curved objects, and a study of sensor systems.

Our  past  work on visual feedback [Gill 1972] in stacking blocks and
inserting objects  in  holes  is  directly  relevant  to  vision  for
assembly.  The  issues  involved  are:  coordinating  hand  and  eye,
utilizing incremental accuracy of vision and  manipulation,  accurate
prediction  to  limit  search,  and  limiting accumulation of errors.
Visual guidance in picking  up  objects  and  stacking  or  inserting
objects  overcomes  errors  in the visual and arm coordinate systems,
and errors in correspondence between the two coordinate systems. This
was  done under realistic conditions, with the hand still holding the
object and in view; the system predicts the appearance of  edges  and
corners  within  a  small  window around the edges to be aligned, and
uses a corner/edge finder to determine alignment error. The arm makes
accurate  incremental  error  correction.  If predictions can be made
accurately, the search window can be small, which increases the speed
of  response  and decreases possibility of confusion with other areas
of the scene. A major limitation to accuracy of  prediction  (and  to
long-term  reliability  in  the  industrial  environment) is drift of
calibration.  That  work   incorporates   calibration   updating   by
self-calibration  to  coordinate  hand and eye coordinate systems and
provide for reliability, as well as to  limit  search  by  accurately
predicting  expected  features. Visual feedback with stereo vision is
much more powerful than without; we have demonstrated  stereo  visual
feedback in a program [Binford unpublished].

We  have  a  set of visual modules to be used in a goal-directed way.
These modules  answer  questions  which  arise  in  practical  visual
feedback  tasks,  and  in analysis of outdoor scenes. Are there color
edges in a selected window? Are there edges in  brightness?  What  is
the  color  of  the  region  in the window? Several preliminary color
region analysis programs have been developed  [Binford,  unpublished,
Bajcsy  1972].  An  edge  operator  [Hueckel 1969] and a region-based
corner/edge finder [Gill 1972] both operate on small windows  of  the
scene,  as  directed  by  a higher level. An edge verifier is used to
confirm or reject edges  predicted  from  context  [Tenenbaum  1970].
Color  identification has long been demonstrated. Recently, a program
has been implemented [Thomas unpublished]  based  on  Land's  Retinex
model  of  color  constancy in vision, which defines a color triangle
from  ordering  of  brightnesses.  The  center  of  the  triangle  is
designated  as  white. The program shows a color constancy similar to
humans; table top scenes are assigned colors reasonably well under  a
variety  of  lighting  with different spectra: sunlight, flourescent,
and incandescent.

Understanding outdoor  scenes  is  a  more  complex  task  than  many
industrial  visual  operations,  but  it  has  utility  for planetary
exploration; it is also  a  useful  exercise  for  developing  color,
texture, and stereo operations. Bajcsy analyzed outdoor scenes with a
combination of color and texture region analysis [Bajcsy 1972]; depth
cues  were taken from the gradient of texture. An attempt was made to
obtain  consistent  interpretations  by  making   three   dimensional
interpretations, and confirming boundaries by independent properties.
Natural objects and natural scenes have been used for our  work  with
motion    parallax    [Nevatia    in   draft]   and   stereo   vision
[Quam,Hannah].These are complementary techniques of depth  perception
which  are applicable for planetary exploration and industrial tasks.
Motion  parallax  can  be  used  for  planning  of   navigation   and
characterizing  large scale terrain features, since it provides large
baselines.

Another aspect  of  vision  with  direct  application  to  industrial
enviroments  is  explored  in  our  work with curved objects. Current
programs use depth information to make useful descriptions of objects
like  a  doll,  a toy horse, a snake, or a glove [Agin]. Programs can
segment these objects into parts like legs, arms or fingers, and link
these   parts   together  to  form  descriptions  of  entire  objects
[Nevatia]. These operations are precursors to a recognition  process.
The  work  depends on a new representation of shape [Binford 1971] in
terms of volume components and a part/whole  segmentation.  Primitive
parts  are  described  by  an axis which is a space curve, and normal
cross sections along the axis.  The  representation  is  based  on  a
formalism,   "generalized   translational   invariance".   There  are
significant theoretical reasons why this representation  leads  to  a
natural choice of parts by simple procedures, in distinction to other
suggested  representations.  We  developed  a  laser  ranging  device
[Binford  1970,Agin]  which  was  operational  two  years  ago. It is
particularly interesting that the device adds a cost  of  only  about
$400 to the cost of a visual system.

A study of visual and depth ranging sensor has been carried out which
provides a background for choosing sensors and interface, and  making
cost  estimates  [Binford  1972, Earnest 1967].  We have maintained a
calibrated hand and vision system for several years [Sobel 1970], and
have  developed  self-calibration  techniques  with  one and two eyes
[Yakimovsky unpublished, Hannah 1972, Gill 1972].

A system for understanding scenes of blocks  from  single  views  has
been  implemented  [Falk  1970]; it segments objects, matches against
internal  models,  predicts  appearance  from  its  assumptions,  and
searches  for  verification  and contradiction based on the predicted
image. Support and spatial relations are  also  obtained.  There  has
been  past  work  on  programs  to  obtain  line drawings as input to
recognition systems. An earlier system was based on  limited  context
of expecting continuation of edges at vertices and predicting missing
edges in certain expected configurations [Grape 1970]. A more  recent
approach has been to use models within the process of generating line
drawings, so that the most efficient  next  guess  can  be  made  and
context used intrinsically in the formation of line drawings.

VISION FOR ASSEMBLY

This  is  a  key part of the proposed project. We propose to evaluate
and implement visual facilities  required  by  a  robot  with  simple
vision, in conjunction with force and touch. These facilities must be
modularized by working out several examples, and made fast enough and
small  enough to be practical for a minicomputer-based robot. We have
considerable experience with such standardization,  and  with  visual
feedback  techniques  in  the domain of blocks. We are now at work on
extending visual feedback techniques to typical  assembly  tasks.  We
are  also  evaluating  methods of automating visual feedback. We have
well-developed systems for modeling objects from the console  and  to
predict  their  appearance  using  techniques  from computer graphics
[Baumgart 1972a,b]. We can predict curved edges from our  models.  We
will use simple region analysis programs and edge operators to locate
curved edges  predicted  within  a  small  window.  Automated  visual
feedback   would  combine  with  evolving  strategy  programs  in  an
interactive system, Assembly Robot Management System, ARMS.  We  will
maintain a calibrated system and improve calibration updating so that
reliability is maintained at a high level.

We have developed a special case program for locating the  pump  body
in  the  pump assembly and moving it to the standard location. We are
at work on several other such tasks involved  in  the  pump  assembly
which  contain  components  of  general  utility for visual feedback.
These tasks are typical of the simple visual operations which we will
implement or adapt.

VISION RESEARCH

While  continuing  research activities are not to be supported by the
proposed project, in all likelihood, the research will suggest useful
programs for the assembly system, particularly work in recognition of
complex objects using depth data.   The  computer  now  has  symbolic
internal  models  of  objects.  We are now in a position to recognize
objects such as a glove, a hammer, etc.   We  will  immediately  deal
with matching descriptions for recognition.  The important extensions
we wish to deal with are occlusion  and  access  to  a  large  visual
memory.   These  will  involve  matching  of partial descriptions and
encoding of partial descriptions as  indices  to  a  small  class  of
similar  shapes,  which  can then be compared in more detail.  We are
working toward making the hardware stable enough that it can maintain
calibration,  and  software streamlined so that the depth measurement
system can be integrated with other hand/eye facilities.   We  intend
to  apply large parts of our programs for description and recognition
of curved objects using only monocular intensity information. This is
a  much harder problem, but within our representation, shares much of
the same code, and relies on  the  same  conceptual  basis  of  local
three-dimensional models.


There is a growing  effort  on  navigation  of  vehicles  in  outdoor
scenes.  While  this  is  not  directly  relevant  to  the industrial
environment, it does  reflect  the  growing  involvement  in  complex
visual  tasks,  and will lead to further development of an integrated
visual system. One approach which has already given  initial  results
[Yakimovsky] is to combine region growing with interpretation using a
world model.  Thus, regions are combined not simply on the  basis  of
uniformity but because combining them improves the interpretation. At
the same time, we will be combining more facilities into these visual
systems, and more usable knowledge in the world model.

	C. MANIPULATOR DEVELOPMENT

Manipulator   research  began  at  Stanford  Artificial  Intelligence
Laboratory  in  1965  with  the  acquisition  and  programming  of  a
prototype  Rancho Orthotic Arm.  A later version of this arm was used
in vision directed block stacking programs.

Our present arm, the result of a continuing design project,  replaced
the  Rancho Arm in 1970. This arm like its predecessor was controlled
by a digital servo program running under the time  sharing  computer.
The new arm was used in the "Instant Insanity" demonstration in 1971.
A second arm of the same type,  with  only  minor  modifications,  is
currently  being interfaced to the computer.   Both arms will share a
common work space and will  be  able  to  be  used  individually,  or
together for two handed tasks.

Each  joint  of the arm is calibrated in terms of torque versus drive
level and it is possible to predict the joint drive levels to produce
a  given  force at the hand. Further, for the hand to exhibit degrees
of freedom of motion, various joints are selected to be "free",  that
is  to have no position feedback applied to them. By exerting a force
at the hand and providing that certain degrees of freedom exist it is
possible  to  perform  a  much  wider  class  of  tasks  than  simple
positional control, for instance screwing in  screws,  mating  parts,
inserting tools and turning cranks.

Touch  sensors on the fingers allow the hand to sense objects without
disturbing them.   When activated the arm will stop when any  of  the
sensors  touch.  Branching arm instructions can test the state of the
sensors and modify the execution of an arm program. When  screwing  a
nut  onto  a  bolt, for instance, the hand is first centered over the
bolt by touching it.

Control of the arms is by a series of high level instructions;  these
instructions  may  be executed directly, or may be assembled together
into an arm program. Arm programs may then be executed,  as  desired,
by  the  control  program.   The control instructions may be accessed
directly by other programs, using the message procedure mechanism, or
in the form of symbolic source file.

Positional  accuracy  at  the  hand  is  typically  0.1  inches  with
repeatability of 0.02 inches. Force control at the hand is within  10
ounces. The data rate to run the arm from an assembled arm program is
of the order of 100 bits/second.

Manipulator research moves  ahead  simultaneously  on  three  fronts:
control,  primitives  and  devices.      Tasks are translated into an
existing hand language and where the task cannot be  accomplished  in
terms  of  existing  primitives,  new  primitives are formulated in a
consistent manner.   When a new primative is  formulated,  real  time
routines  are  developed  to perform the action.  This also leads, in
many cases, to the development of new  hardware.        Also  as  new
developments  occur  in  hardware design, existing real time routines
are either re-written or modified.

Research is now concentrating on the following areas:

Strain gauge instrumentation of the wrist, in order to allow for more
sensitive  interaction  of  the  hand with a task.   High resolution,
high sensitivity touch sensors, to increase tactile  recognition  and
control  abilities.      These  developments will lead in turn to new
control concepts.

Development of tools for the hand. These tools  can  be  instrumented
such  that  the  computer can feel and or see at the end of the tool.
This should be a very important area of research as  it  goes  beyond
present human capabilities.

Two handed arm control, the planning and execution of tasks requiring
the coordinated motion of both arms. This will require  an  extension
of  the  existing  servo  program and the analysis of synchronization
problems. The development of collision avoiders, so  that  arms  will
not collide with other objects and with each other.

Modification  of  the  existing  arm  design  to  provide  for better
incremental motions, more degrees of freedom and a more rugged design
suitable  for  industrial  use.   Development of arm and eye computer
interfaces.   These contain  synchronized  analog/digital  converters
for input and current controlled drivers for output.

The  development  of  new hands is expected as more complex tasks are
attempted.   At present, with only one hand, we have been  performing
the  positioning  type  of  hand  actions.    When  the second arm is
operational we will be able to perform holding and pulling actions as
well.

	D.  SYSTEM REQUIREMENTS FOR ADVANCED AUTOMATION

A  key  element in the development of any large programming effort is
the software support.  There has been continual  effort  along  these
lines  at  the  Stanford  Artificial Intelligence Laboratory and many
useful ideas have been developed.  We propose to extend the  research
along  two  separate lines:  support for research and system software
for operating automated systems.  We will briefly describe  our  past
efforts in this area and what we propose to undertake.

The  Stanford  Artificial  Intelligence  Laboratory  has pioneered in
several areas of system programming, including  timesharing  sys36700
interactive graphics, and programming languages.

A  great  deal  of  additional effort has gone into direct support of
hand-eye research.   Very early, we established  a  set  of  standard
formats  and  library routines [Pingle 72a].  A programming language,
SAIL[Swinehart], was implemented in 1968.  The language  incorporates
special   features  for  large  real-world-oriented  problem  solving
systems  and  is  currently  in  use  at  a  number   of   Artificial
Intelligence and other laboratories.

As  we began to assemble larger and more sophisticated collections of
programs, we felt a need  for  flexible  mechanisms  for  cooperating
sequential  processes.   The  resulting monitor and language features
[Feldman & Sproull 1971] have  worked  well  for  us  and  have  been
influential   on  other  projects.     Recently,  we  have  added  an
additional set of features to SAIL  to  facilitate  non-deterministic
and  event-directed  programming[Feldman 1972].   The early reactions
to this work, both within the laboratory and without  has  been  very
encouraging.

We  expect  relatively  little  work  to be needed on further systems
support for basic research.   There will undoubtedly  be  efforts  to
take  advantage of new hardware and of new programming ideas that may
develop in the course of this program.    Almost  all  of  this  work
should be covered by other funding.

The  development  of  software  for the assembly station will require
more  of  our  effort.    The  central-local  computer  configuration
requires  code to be resident in the local computer and in the shared
machine.    The low level programs in the local machine will  not  be
modified  often,  thus  we can assume that a compiler resident in the
central machine will suffice.   The  planning  programs  and  general
vision programs will be resident in the shared system.  There are two
possible configurations for the shared computer system:     A  PDP-10
equivalent  machine  or a software package in ASA Standard Fortran IV
which will run on any sufficiently large machine.    One of the tasks
under this proposal would be to evaluate these two alternatives.

A major share of the programming research effort will be concentrated
on developing a programming system for automated assembly.  This will
combine  ideas  from  our  past  work  on  languages, representation,
computer graphics and arm control.  We view this task as a branch  of
the  larger  effort on automatic programming being undertaken in this
laboratory and elsewhere.


III. UTILIZATION PLAN-DISSEMINATION  OF  RESULTS  AND  RELATION  WITH
INDUSTRY


There are several other efforts, at varying levels of sophistication,
in the area of computer controlled advanced automation.  The Stanford
Artificial Intelligence Laboratory is the oldest and largest of these
and  has  provided  the  cutting  edge  of much of he development. We
intend to continue this role and to lead  in  the  development  of  a
practical   general   purpose  assembly  system  and  the  associated
software.

All of the discoveries made under this grant will, of course, be made
public.    We view our role in information transfer in a much broader
way.  Having no desire to sell products and services and  no  clients
with   interests  to  protect,  a  university  laboratory  should  be
primarily a place for acquiring and spreading knowledge.

The Stanford Artificial Intelligence Laboratory has always considered
information transfer to be one of its primary goals.  In addition, to
our published papers and reports, we  have  made  a  number  of  film
reports  which  have  been  extensively  circulated.  There have been
thousands of visitors, ranging  from  elementary  school  classes  to
major  scientific,  governmental  and  industrial  missions from many
countries. We have also encouraged extended visits  by  workers  from
other laboratories and from industry. Members of the laboratory staff
have played a major role in the organization of journals, conferences
and  seminars  in  artificial  intelligence.  Many of our algorithms,
systems, and hardware designs have been adopted by other groups.

The  success  of  the  program  proposed  here  depends  heavily   on
interaction  with  industry.   We will need additional information on
the opportunities for automation and will try to involve manfacturers
in  the  development  of  the  pilot assembly station.   We have good
working relationships with a number of firms and  we  expect  that  a
concrete project will help forge the bonds necessary for the transfer
of advanced automation technology from the laboratory.

One promising approach to technology transfer is a  staff  associates
program.   This  involves residence of about a year's duration at the
laboratory by  industrial  scientists.    Such  a  program  was  used
successfully at the MIT, Electronic's Research Laboratory in the area
of programming systems for numerically controlled  tools  and  design
automation.    We  have initiated a similar program in the laboratory
(cf Appendix A) and are pursuing additional forms of cooperation.

			Appendix A

	The  Stanford  A.I.  Lab  has  been  engaged  in  research on
advanced automation technology.  Our work has progressed to the stage
where  we  are  seriously  engaged  in  developing  and disseminating
practical applications.   To do this effectively, we  will  need  the
help  of industrial organizations with computer sophistication and an
understanding of practical problems.  One form of  cooperation  which
we  are  engaged in is a staff associates program.  A staff associate
is a member of another organization in residence at the A.I. Lab.  We
have  had  a  number  of staff associates over the years, mostly from
foreign countries. The enclosed letter from Japan is typical  of  the
inquiries  we receive.  The purpose of the new program is to invite a
few domestic inductrial organizations to submit  candidates  for  the
staff associates positions.

	We  expect  to be able to have three to five staff associates
in the laboratory at a time.  The associate would remain for about  a
year and would be paid by his sponsoring organization.  He would be a
regular  member  of  the  laboratory  staff,  working  on  automation
problems.

	There  are,  of  course,  academic programs at Stanford which
could also involve work at  the  Artificial  Intelligence  Lab.   The
staff  associate  program  differs in that the selection criteria are
different, the timing more flexible, and the expected time devoted to
course  work  much  less.   We  believe that it is the most effective
mechanism for us to learn from and help industrial  practitioners  of
computer-aided advanced automation.


			Jerome A. Feldman